Lecture 10: Semi-Markov Type Processes

The KTH Visit in Semi-Markov Processes

Asymptotic Behavior of Multivariate Reward Processes with Nonlinear Reward Functions

Solving Generalized Semi-Markov Processes using Continuous Phase-Type Distributions

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...

Solving Generalized Semi-Markov Decision Processes Using Continuous Phase-Type Distributions

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...

Estimation in a Proportional Hazard Model for Semi-markov Counting Process

Estimation is studied in a regression model for counting processes whose baseline intensity processes are of semi-Markov form. Asymptotic normality is established for a Breslow-type estimator of the cumulative baseline hazard for each gap time of the counting process.